Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clean up fugaku files and gromacs builds with gcc now with minor spac… #161

Merged
merged 1 commit into from
Mar 7, 2024

Conversation

jdomke
Copy link
Collaborator

@jdomke jdomke commented Mar 6, 2024

…k modifications to fujitsu-mpi package

@github-actions github-actions bot added the configs New or modified system config label Mar 6, 2024
Copy link
Collaborator

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to check: you can build benchmarks on Fugaku using these changes? A there are a couple spots that look like they might cause problems, but fixing those could be reserved for a later PR if this works for you.

@@ -128,9 +53,19 @@ compilers:
cxx: /vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/bin/g++
f77: /vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/bin/gfortran
fc: /vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/bin/gfortran
flags: {}
flags:
ldflags: {"-lelf -ldl"}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(potential issue) Des this appear to concretize for you? I'm not familiar with this YAML syntax (it looks like a set of one element), typically one would write:

ldflags: "-lelf -ldl"

Also, would you be able to mention the error you get without this? I don't want to hold this PR up on it, but I may be able to make build suggestions that avoids the need for this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linking against fujitsu's blas lib (SSL2) with gcc results in missing symbols, which is why i need those flags

providers:
mpi: [fujitsu-mpi, openmpi, mpich]
blas: [fujitsu-ssl2, openblas]
lapack: [fujitsu-ssl2, openblas]
scalapack: [fujitsu-ssl2, netlib-scalapack]
fftw-api: [fujitsu-fftw, fftw, rist-fftw]
fftw-api: [fujitsu-ssl2, fftw, rist-fftw]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not have an effect: fujitsu-ssl2 does not provide fftw-api. Spack can work around this because the expressions here are taken as preferences by spack (i.e. not hard requirements). In other words, this shouldn't cause anything to fail, but I also think it isn't necessary for anything to succeed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

afaik, fujitsu-ssl2 provides fft functions, but for some reason the fftw.h header is missing. i'm also not yet 100% sure how to set it properly, when i compile (forgot) gromacs or hyper i see that it doesnt find the header and then builds fftw from source (which isn't the best approach, IMHO). i don't think 'fujitsu-fftw' exists, so the previous config is definitely wrong.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is that because Spack's builtin fujitsu-ssl2 package.py does not have provides(fftw-api) spack will never assume it is useful as an fftw provider and will always try to use something else (the logs generated by ramble workspace setup should include the concretization, and I assume Spack will have chosen something besides fujitsu-ssl2 wherever an fftw implementation is needed).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks for the clarification

compiler-fujitsu:
spack_spec: fj@{default_fj_version}
compiler-clang:
spack_spec: clang@{default_llvm_version}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(no change needed) This may be required until we invoke Ramble in a way that avoids having it look for additional compilers.

OMPI_CC: /vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/bin/gcc
OMPI_CXX: /vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/bin/g++
OMPI_FC: /vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/bin/gfortran
OMPI_F77: /vol0004/apps/oss/spack-v0.19/opt/spack/linux-rhel8-a64fx/gcc-8.5.0/gcc-12.2.0-sxcx7kmt3qiktffgzzvrj2wmup3g32bc/bin/gfortran
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(potential issue) When we set OMPI_* variables, it is typically to point them at the underlying spack wrappers, it might be preferable to handle this as an update to fujitsu-mpi's package.py file in Spack.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i understand. i can take a look how to fix fujitsu-mpi with an additional PR. for now its just to work around problems i run into with fugaku.

@jdomke
Copy link
Collaborator Author

jdomke commented Mar 7, 2024

I wanted to check: you can build benchmarks on Fugaku using these changes? A there are a couple spots that look like they might cause problems, but fixing those could be reserved for a later PR if this works for you.

yes, i was able to build amg2023 and gromacs with gc with those changes. i'll check clang next, and verify that i didnt break anything for fujitsu compiler.
but i had to move to gcc because gromacs was refusing to build with fujitsu (the c++17 support is not 100% working and result in linker errors with some ABI 11 functions, don't ask me why, but for now gromacs+fujitsu is a no-go)

Copy link
Collaborator

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mentioned this working for you so I think the best approach is to merge this as is and address any concerns in a follow-up.

Thanks!

@scheibelp scheibelp merged commit 5fcf954 into LLNL:develop Mar 7, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
configs New or modified system config
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants